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ABSTRACT 


ITM students are the Department of Defense (DoD) Information Technology (IT) experts 
of tomorrow. Their ability to use the Internet is an integral part of their future job 
performance. This study examines the current state of Internet usage among Information 
Technology Management (ITM) students at the Naval Postgraduate School (NPS). T his 
study uses direct observation to determine ITM students’ ability level in performing basic 
Internet tasks. The results of these observations are used to show trends and changes in 
ability between ITM sections. These trends are used to draw conclusions as to the 
methods and sources currently used to instruct ITM students in Internet usage. This study 
shows a need for more formalized Internet instruction at NPS. Recommendations are 
made for both tutorials and classroom instruction. Future research into the effectiveness 
of new instructional methods is also recommended. These results and recommendations 
are applicable for export to all DoD Internet users. 
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I. INTRODUCTION 


A. PURPOSE 

The purpose of this thesis is to examine the current state of Internet usage by 
Information Technology Management (ITM) students at the Naval Postgraduate School 
(NPS) in Monterey, California. The current state of usage, for the purposes of this thesis, 
consists of the student’s ability to use the Internet along with the current instructional 
sources which teach the students this ability. 

ITM students will become DoD’s information technology managers of the future. 
It is assumed that there is some advantage to having DoD IT managers who are well- 
versed in using the Internet. This thesis will examine these theoretical advantages of 
Internet usage, and help ensure that ITM students are being prepared to use the Internet 
in the future. 

This thesis will not solely benefit ITM students, however. All graduates from NPS 
should be literate in the use of the Internet. The results of this study will be applicable 
to all graduates of NPS as well as other Internet users through out DoD. 

B. RESEARCH QUESTIONS 

There are two primary research questions in this thesis. One looks at the ability 
of ITM students, while the other looks at the current source of Internet instruction. 

Is there a relationship between experience and the ability to use the Internet 
effectively? It would seem obvious that the ability of a student to perform a given task 
would be proportionate to the experience or knowledge level of that student. This 
research question will confirm this fact. It will also, however, provide a measure of 
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ability at various points on an experience timeline. This should be useful in deter minin g 
methods to improve the learning process, if necessary. 

What instructional source has taught ITM students to use the Internet? In other 
words, where are the students learning how to use the Internet? The students may be 
learning from three different categories of sources. They may be learning from informal 
sources such as their fellow students or by exploring on their own. They could be 
learning from formal instmction such as class assignments, or they could be learning from 
semi-formal sources such as computer center classes or personnel. This is of interest, 
especially if the prior research question shows that changes are necessary. 

C. SCOPE 


This thesis will consist of an observation of a sample set from the ITM student 
population to answer the primary research questions. Five students from each of the four 
present ITM sections, PM33, PM41, PM43, and PM51, will be used as the sample set. 
Students will be observed performing a series of tasks using their Unix accounts in one 
of the NPS computer center labs. Their ability to perform the task will be noted. The 
time required will also be recorded. The methodology of this observation process is 
discussed in much greater detail in Chapter H. 

The information gathered in these observations will then provide the basis for the 
conclusions and recommendations made in this thesis. 

D. LITERATURE REVIEW 

1. Learning Theory 

A short overview of learning theory and its application to computer based tasks 
is important to the overall understanding of why the students’ performance did or did not 
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meet the expectations of the researcher. It is also important to the development of ways 
to improve upon current instructional conditions. 

There are two parts to describing the learning process. First, there is the type of 
knowledge with which we are dealing, and second, there is the process of knowledge 
instruction 

2. Knowledge Types 

There are three basic categories of knowledge: declarative, procedural, and self- 
knowledge. (Dillon & Sternberg, 1986) 

Declarative knowledge is the raw facts associated with a subject. It is usually a 
hierarchial elaboration on a given subject, with the most detailed information contained 
at the very bottom of the structure. There is no need to use the knowledge, only to be 
able to present it. An example of declarative knowledge would be the rote recital of dates 
on a history examination. 

Procedural knowledge is knowledge which gives the ability to perform some 
function. It is the natural follow-on to declarative knowledge. Procedural knowledge 
requires more sophisticated reasoning on the part of the user, as the user must develop 
different sets of logic for different tasks. 

Self-knowledge is an awareness of one’s reasoning process. It involves accessing 
the logic used in procedural tasks and examining it for strengths and weaknesses. This 
is the most advanced form of knowledge. 

Procedural knowledge was most relevant for this study. It allowed the students 
to perform the given tasks on the Internet. 

3. Knowledge Instruction 

There are four phases to knowledge instruction (Alessi & Trollop, 1991): 
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1. Presenting information. 

2. Initial acquisition. 

3. Fluency and retention. 

4. Assessment. 

Presenting information and initial acquisition normally take place at the same time. 
In a classroom, for example, an instructor presents the information at the same time as 
a student begins to retain it. This can be accomplished either by classroom instruction 
or by tutorials. This corresponds to the learning of declarative knoAvledge. 

Fluency and retention require the use of drills which cause a student to work 
through a problem on his or her own. Success provides the procedural knowledge that 
students must use to complete the tasks in this thesis. 

The last area of knowledge instruction is the assessment or testing of ability. This 
provides validation to both the instruction method and the student’s ability. 

In general, computer based learning is an automated application of the techniques 
already in use manually, especially tutorials. The current advantage of computer usage 
is increased speed and attention-holding capability. (Ngwenyama, 1993) 

The phases of instruction of most interest to this study are fluency and retention, 
and assessment. Assessment will help determine the current state of the ITM student 
population and fluency and retention will help develop ways to improve it. 

4. Current Benefits of Internet Usage 

The benefits of Internet usage are central to why this study was conducted. 
Therefore, we must look at what the Internet is and why its use by ITM students is 
important. 

The Internet is the largest computer network in the world, primarily supported by 
government grants. The origin of the Internet was the DoD’s Advanced Research Projects 
Agency Network (Arpanet) begun in the late 1960’s. In 1983, the Transmission Control 
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Protocol/Intemet Protocol (TCP/IP) was made mandatory by the Defense Communications 
Agency for both the Arpanet and the Milnet, a DoD network. After this point, 
commercial enterprises began to produce products to this standard. The collection of over 
17,000 networks that now share this protocol is known as the Internet. The largest 
backbone of the Internet is now the National Science Foundation’s NSFnet with over 
4,000 networks linked to it. In 1992, the Internet had over 1 million host computers with 
many millions of users.(Perry, 1992) 

The benefit of using the Internet to find and retrieve information is a more 
productive group of students. Students with an ability to access information on the 
Internet are not tied to geographic locations, such as an institution library. Nor are they 
excluded from access to information due to academic position or standing. (Watson, 1994) 
However, using the Internet is not intuitive. Since the Internet is decentralized, 
there are no common standards and no formal advertisement of information locations. 
The Internet is far too large for the imposition of a hierarchical structure. (Schwartz, 
1993) 

As an example to illustrate the need to provide efficient locating capabilities for 
Internet information, in December 1991, the University of Colorado conducted a study 
involving the monitoring of File Transfer Protocol (FTP) traffic. During the course of the 
study (13 days), 39,324 files were transferred. However, this was only 44 percent of the 
actual FTP traffic. The remaining traffic was directory requests, or in other words, 
individuals looking around for information. (Schwartz, 1993) 

What is needed is a set of services which allow the user to search and retrieve the 
various information available on the Internet. This will allow students to achieve the full 
potential of Internet usage. (Obraczka et al, 1993) 


5 



E. BASIC INTERNET TOOLS 


As previously discussed, the decentralized nature of the Internet means that 
information may be found at many different sites, and, possibly, in several different 
formats. The location of information may or may not be published, or known to the user. 
Users are often required to use a variety of tools to search for and acquire desired 
information, such as text files, current research material, and colleagues e-mail addresses. 
An understanding of theses tools is needed prior to conducting the observation of ITM 
students attempting to perform various Internet tasks. (Schwartz, 1993) 

Table 1 shows a basic set of tools that may be used in the Internet. It is not 
intended to be an exhaustive list, but instead as an overview of current capabilities. 

Electroii i c Mail . Simple Mail Transfer Protocol (SMTP) allows the user to send 
mail across the Internet regardless of the user’s own particular file system. The Internet 
provides a series of relay mail servers which use the mail’s Internet Protocol(IP) address 
as a means to determining the mail’s ultimate destination. Mail is normally short 
messages in ASCII text. ASCII text is generic text that has been stripped of program 
specific control characters. (Postel, 1982) 

Netfind. The use of electronic mail relies on knowledge of a recipient’s e-mail 
address. Netfind locates unknown e-mail addresses. It uses the search mechanisms 
available within each individual domain’s network mail server rather than a common 
format that would be needed for a central database. 

To begin a Netfind search, a user provides a desired person’s name and a short 
group of keywords describing the possible location of the desired person. An example 
of such a string could be "smith navy mil." Netfind then uses this information in a seed 
database which contains a listing of possible domain matches. It then asks a user to pick 
the domains he or she desires to search. In the example, Netfind may return 
"nps.navy.mil", "usna.navy.mil", and nosc.navy.mil." Once the user chooses a desired 
domain, Netfind queries the SMTP server as to all possible mail forwarding information 
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Internet Tool 

Electronic Mail (E-Mail) 
Netfind 

FTP (File Transfer Protocol) 
Gopher 

Veronica 

WAIS (Wide Area Information 
Servers) 

Telnet 

WWW (World Wide Web) 

Web Robots 


Table 1. Basic Int 
After 


Use 

Electronic communication 
Locate E-Mail addresses 
Transfer of files 

A public access distributed document 
retrieval system 

Keyword searching of document titles 
at Gopher sites 

Full-text searching of distributed 
documents 

Access to remote machines 

A public access distributed hypertext 
and multimedia document retrieval 
system 

Keyword searching of pages at WWW 
sites 


let Tools and their Uses 
atson, 1994 
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on each machine at that domain. Netfind uses the "finger" command to return all possible 
e-mail addresses for the desired individual on each machine. 

The seed database constantly updates itself from a variety of e-mail monitoring 
and reporting services. Since 1993, Netfind could search over 9,000 sub-domains, 
containing the e-mail addresses of approximately 5.5 million users. This includes 191 
sub-domains in the ".mil" domain. (Schwartz, 1993) 

FTP. File Transfer Protocol allows a user to transfer files between two computers. 
The most common use on the Internet is anonymous FTP. In anonymous FTP, the user 
has access to a computer system without having an account. This allows any user to 
obtain a large amount of information, without the usual account overhead. (Rovers, 1994) 
Gopher. Gopher is a hierarchical structure that allows users to retrieve distributed 
information on the Internet. Gopher organizes information into a series of directories and 
sub-directories, with the lowest level being the actual files. Users navigate through 
gophers using a hierarchical tree structure. (Obraczka et al, 1993) 

Veronica. Veronica does keyword searches on a database of titles of gopher items, 
both files and directories. The Veronica service performs its function in two parts. First, 
menu data for constructing the database is gathered from registered gopher servers. The 
listing of registered gopher servers is maintained at the University of Minnesota. Second, 
Veronica allows users to conduct keyword searches on this database. These two functions 
are not necessarily performed by the same machine; the database may be obtained from 
a common repository. (Foster & Barrie, 1993) 

WAIS. Wide Area Information Servers is a full-text information retrieval 
architecture. It works by having a central "Directory of Services" which points a user to 
the database that contains the information they desire. 

To conduct a WAIS search, a user prepares a query which is transmitted in the 
WAIS protocol to the "Directory of Services," or yellow pages, which is installed on a 
known WAIS server. The "Directory of Services" forwards the query to all relevant 
databases, based on its knowledge of what these databases contain. A specific database 
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then returns the user a ranked listing of all objects which contain the desired text string. 

Each individual database is responsible for providing updated content information 
to the central yellow pages. Currently there are over 300 databases referenced by the 
"Directory of Services." (Obraczka et al, 1993) 

World Wide Web . The World Wide Web was developed at CERN (European 
Laboratory for Particle Physics) in Switzerland. It uses a distributed structure supported 
by a graphical user interface(GUI) using hypertext techniques. Data is organized into text 
objects which are linked to a series of hypertext cover sheets or index objects known as 
Web-pages. (Obraczka et al, 1993) 

Web Robots . A Web Robot explores the Internet autonomously to find new 
information. It does this by starting with a known WWW document and identifying new 
places to explore by looking at the links from that document to other documents. It then 
repeats the process indefinitely, while compiling indices of the information it 
encounterers. When a Web Robot is sent a query by a user, it uses these indices to 
provide a listing of relevant WWW sites. (Pinkerton, 1995) 

For convenience, these tools can be divided into three categories: 

1. Those which allow the Internet user to communicate, such as e-mail. 

2. Those that allow the user to transfer or view information, such as FTP, Gopher 

or WWW. 

3. Those that allow the user to find information, such as Netfind, Veronica, or 

WebCrawlers. 

The most complex tools are the ones associated with the finding of information. 

With a working knowledge of this basic set of tools, the average Internet user 
should be able to find and access a reasonable portion of the available information on a 
given subject. However, no published articles or studies were found as to what portion 
of the total available information any one tool was actually able to locate. 
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n. METHODOLOGY 


The methodology of this thesis is an observation of students performing a set of 
tasks. The organization of this methodology is derived from Emory & Cooper, 1991. 

A. DESIGN STRATEGY 

This thesis portrays the ability of Information Technology Management (ITM) 
students to use the Internet effectively. The desired data should provide both a concrete 
measure of each section’s ability and the means for comparison between the sections. A 
section is an administrative designation used at NFS to , denote a group of students 
curriculum and arrival date. For example, PM33 is the ITM section which arrived in the 
third(3) quarter of fiscal year 1993. The following set of hypotheses will be used as a 
basis for the observations conducted: 

Hypothesis 1. The ability to perform various tasks using the Internet and student 
Unix accounts improves with experience, experience being defined as time at the Naval 
Postgraduate School. 

Hypothesis 2. The prime source of the students’ ability to use the Internet and 
their Unix accounts is not through a formal source, such as class assignments, computer 
center lectures, or computer center personnel, but rather through an informal method of 
sharing knowledge among themselves. 

A set of direct behavioral observations was chosen to test hypothesis 1. Direct 
observation was chosen in view of the observer’s limits in time, money and hardware. 
An unobtrusive method such as automatic monitoring of activities within a student’s Unix 
account would have provided better data, but this method proved physically unpracticable 
within the scope of this study. 

Three categories of tasks were used as a representative measure of ability as 
contained in hypothesis 1. These three categories were communications, data transfer and 
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knowledge acquisition. Time required to perform the task was used as a measure of 
effectiveness in each case. Although the quickest answer does not necessarily provide the 
best answer, it was the most suitable measure in this case. This choice of measurement 
is discussed further below. 

a. Communications: Was the subject able to communicate effectively using e- 
mail, including the acquiring of various unknown e-mail addresses? 

b. Data Transfer: Given the location of specific pieces of data, could the subject 
retrieve them using a variety of tools, such as file transfer protocol (FTP), gopher and the 
world wide web (WWW)? 

c. Knowledge Location: Given a specific topic or site, could the subject locate 
the requested information? 

Hypothesis 2 would be tested in the second portion of the observation, using a 
short series of demographic questions. 

B. MEASUREMENT 

Along with discussing the reasons for the various scales used within the 
observation, this section looks at the validity, reliability, practicality, and error sources 
associated with the measurements used. 

1. Scale 

A ratio scale was used for the observation of the tasks associated with hypothesis 
1. As discussed in the design strategy, the amount of time required to perform a given 
task was recorded. Any task requiring more than five minutes was scored as "unable to 
perform the task." The time taken was recorded by the observer. 

An ordinal scale was used for hypothesis 2. Students were asked to rank the top 
three sources from which they had gained the knowledge they had used in the task-based 
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portion of the observation. Although this ordinal scale will not allow parametric testing 
of this data, it is the most appropriate form in which to measure this data. Independent 
tracking of a source of knowledge was not possible. 

2. Validity 

Vahdity is the ability of the observation to measure what it is designed to measure. 

Content, criterion, and construct validity are the primary concerns. 

a. Content validity is the degree to which the observation covers all applicable 
areas of the material being studied, in this case Internet ability. Based on the literature 
review and the pilot test, there seems to be good content validity. The three categories 
chosen to test hypothesis 1, communications, data transfer, and knowledge acquisition, 
appear to provide adequate coverage of the tasks normally required of a student Internet 
user. Hypothesis #2 has a very narrow set of possible sources as to where the student 
acquired his or her skill level. The ordinal questions chosen should provide a complete 
set of possible answers. 

b. Criterion validity is how well the chosen measure, in this case time, represents 
the actually state of what is being measured. Using time as a measure for hypothesis #1 
is not the only possible choice. A measure of the quality of the data delivered per task 
may have been viable, or an analysis of the technique and thought processes used may 
also have been of value. However, time is the most useful measure available because it 
provides the ability to conduct statistical analysis plus it eliminates problems with 
observer interpretation. 

c. Construct validity is the degree to which a measure conforms to other 
theoretical work. In this case, there were no other studies with which to compare. The 
tasks the subjects are asked to perform are considered basic and fundamental to using the 
Internet, therefore, the subjects ability to perform these tasks is a valid measure of actual 
ability and should have good construct validity. 
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3. Reliability 


Reliability is the degree to which the observation provides consistent results. 
There are three types of reliability which are of concern: stability, equivalence, and 
internal consistency. 

a. If a subject scores the same on repeated observations then there is stability in 
the measurement. In this thesis, there were no retests of subjects. Therefore there is no 
actual means to test the stability of the measured results. However, there appears to be 
no reason that the measure is not stable. Situational errors may have occurred, as 
discussed below, but they are a function of observation environment rather than the tool 
itself. 

b. Equivalence considers whether different observers will achieve different results. 
Again, there is no means to actually confirm equivalence in the observation since there 
was only one observer. The observer kept interaction with the subjects to a minimum and 
only recorded objective quantities such as time. A second observer should be able to 
reproduce similar results. 

c. Internal Consistency ensures that similar tasks with one subject will provide 
similar results. There was no attempt to test internal consistency and there is no way to 
verify that it exists other than to note whether subjects who performed poorly in certain 
portions of the observation continued to perform poorly and those who performed well 
continued to perform well. 

4. Practicality 

Practicality was a major design consideration due to the previously discussed 
shortcomings in resources, mainly time. The observation was easily carried out by a 
single observer using a public lab in the course of one work week. All tasks are straight 
forward with a single possible correct answer, while allowing individual subjects to use 
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a variety of techniques to reach the answer. The scales used provide a simple, useful data 
set. 


5. Error Sources 

There were several opportunities for errors to have been introduced into the 
measurement process. However, these errors are consistent between sections of the 
sample, and although they may flaw the empirical data, the relative measures of the data 
remain valid. 

a. Respondent error seems to be minimal. All subjects appeared to the observer 
to honestly attempt to perform all tasks. All have similar backgrounds with regard to 
career and social class. There may have been small amounts of error introduced by 
student anxiety over other classes, et cetera, but this did not appear to be significant. 

b. Situational error is the most significant source of error. It was impossible to 
tailor the lab situation to each subject, therefore there were multiple sources for situational 
error. Many of the subjects were used to working on home PC-based interface as 
opposed to a Unix-based workstation. The presence of other students in the lab may have 
affected the subjects ability to concentrate on the given tasks. The presence of an 
observer may have also contributed to an uncomfortable atmosphere in which to conduct 
the experiment. All these factors combined to give an advantage to subjects who were 
used to working on a Unix workstation in a crowded lab environment. 

c. The measurer was a minimal error source. The same observer performed all 
observations. The personal acquaintance of the observer to any of the subjects did not 
flaw the observation, since all tasks were of a very objective nature. 

d. Instrument was also a minimal source of error. The instrument is simple and 
has been shown to be valid. Both the pilot test and the actual testing showed the tasks 
to be clear and complete. 
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C. SAMPLING DESIGN 


Since Hypothesis 1 required various experience levels, a stratified sample of the 
ITM student population was needed. A sample of five students from each of the four 
current ITM sections was chosen as the sample that would provide the needed data. This 
sample size was chosen based on the small total population size. This choice is addressed 
in detail in the discussion of sample size. 

1. Sample Type 

The sample was drawn from the ITM student population at large. It was a 
convenience sample in that the observer had no means of drawing a random sample from 
the population, mainly because there was no way the observer could compel participation 
from the randomly selected subjects. Therefore, the observer relied on volunteers to 
provide the sample set. 

2. Sample Size 

As previously discussed, five subjects were used from each of the four current 
ITM sections. This was necessitated by the small size of the total population. The total 
ITM population by section is as follows: 
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PM33 

27 

PM41 

53 

PM43 

28 

PM51 

57 


Table 2. ITM Population by Section 

Since there was no basis for an interval estimate, the effects of such a small 
sample size on the observation could not be determined until the results were examined. 

Rather than begin with an interval estimate to determine the desired sample size, 
the standard deviation of the sample was used to determine the interval estimate with a 
0.95 confidence level. Using the standard deviation from each section’s sample, the 
interval estimate was determined as follows: 


i=(1.96 )—— 
n/n-1 


where 

i = Interval estimate in minutes, 
s = Standard deviation of the section sample, 
n = Sample size, in all sections this number was 5. 
1.96 = Standard errors for a 0.95 confidence level. 
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This equation gives interval estimate (i) by section with 0.95 confidence level as follows: 



s 

i 

PM33 

0.875 

51 secs 

PM41 

0.909 

53 secs 

PM43 

0.852 

50 secs 

PM51 

0.776 

46 secs 


Table 3. Interval Estimate for Task Observations 

Therefore, with a 0.95 confidence level, the results of the observation of the 
sample set are within 53 seconds of the population as a whole based on the standard 
deviation of each section’s data set and a sample size of five. 

By comparison, the population would have been unable to support the observation 
had interval estimate been used to dictate sample size. An interval estimate of 30 seconds 
increases the required sample size to 37 subjects per section. Even with a finite 
population adjustment, the sample size remains over 15 subjects per section. Given the 
population size this was highly impracticable. 

Note: Since the results were in a bipolar distribution, the standard deviation used 
for each section was taken only from the positive answers and not the "unable to perform 
task responses. That is the observations that could be quantified in terms of time. 
Therefore, the sample mean used to calculate the standard deviation is not representative 
for use as a comparison between the groups, and was only used as a point around which 
to measure the deviation of each section. 
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3. Sample Validity 


Validity is also important in the sample process. In this case, validity is how well 
the sample set represents the entire population. There are two parts to sample validity, 
accuracy and precision. 

a. An accurate sample will have no systemic variances or biases associated with 
it. The use of a non-random sample could be a source of systematic variance. However, 
in this case the standard deviations were low enough to make this unlikely. 

b. Precision is a reflection of how well the effects of chance have been removed 
from the sample set. This usually comes into play when all other systemic variances have 
been accounted for. Again, the low standard deviation of the sample sets shows that there 
is good precision. 

D. THE PH^OT TEST 

A pilot test of the observation tasks was conducted prior to the actual observation 
of sample subjects. The pilot test was intended to show errors in both the observation 
tasks and the observer’s technique. The pilot test procedures were identical to the actual 
observations in controllable respects, such as location, tasks and equipment used. The 
personal relationship between the subjects and the observer provided the only possible 
source of error in this procedure. 

Three volunteer subjects were chosen from PM33, the senior section. The 
volunteers were chosen based on the observer’s opinion that they provided a valid cross- 
section of the PM33 population. There was no attempt to maintain randomness in the 
selection of pilot test subjects. 

Although the pilot test subjects showed varied levels of competence on some tasks, 
their results within each category were very similar. This indicated that the initial group 
of tasks should be sufficient to provide a representation of each section as a whole. 
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In post-observation discussion, all three pilot test subjects indicated that the tasks, 
in their opinions, provided a fair test of their abilities to perform the three categories of 
tasks. They also anticipated a difference in results between sections due to their own 
experience in learning to perform the observed tasks. 

E. OBSERVATION 

The observation was conducted using a Sun Sparc 10 workstation in the public 
computer lab in Root Hall at the Naval Postgraduate School, Monterey, California. The 
subjects used their own accounts and were allowed to use any techniques or tools that 
were available. 
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ni. OBSERVATION RESULTS 


This chapter is a summary of the data collected during the task observations. 
Although, in some cases, this is not the raw data, all cumulative data presented are taken 
directly from the raw data. In all tables, represents a ’0’ for ease of visual inspection. 
The remaining raw data are available in Appendix B. An analysis of these data is 
provided in Chapter IV. 

A. DEMOGRAPHICS 

The following data provide the demographic information that was gathered about 
the subjects during the observation. The demographics concern solely the characteristics 
of the subjects’ Unix account usage. No personal demographics, such as age, sex or 
military community, were recorded during the observation. 

Table 4 is the response to a direct question as to how often a subject logs in to 
his or her Unix account. The question was posed by the observer to the subjects. It is 
the subject’s own answer, without independent verification. The sample for each section 
was five subjects. 



PM33 

PM41 

PM43 

PM51 

< Once/Week 

- 

- 

- 

- 

OnceAVeek 

- 

- 

3 

- 

2-3AVeek 

- 

- 

1 

- 

4-5AVeek 

- 

- 

- 

- 

6-7AVeek 

1 

- 

1 

- 

> Once/Day 

4 

5 

- 

5 


Table 4. Login Frequency 
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The data for Table 4 were not independently gathered or confirmed. There is no 
confidence factor or interval estimate computed for this data. The data, however, still 
provide insights into the usage patterns of ITM students. The interpretation of these data 
will be discussed in more detail in Chapter IV. 

Table 5 shows the extent to which a subject customized his or her computer center 
account workspace. This was recorded by the observer without input from a subject. In 
this case, customization is defined as any modification to the standard Unix X-windows 
workspace that allows a subject to function more effectively. In Table 5 "yes" means the 
account was customized, and "no" means the account was not. 

A hotlist, as used in Table 5, is a personalized list of Uniform Resource Locators 
(URLs) maintained within a World Wide Web (WWW) browser, such as Netscape or 
Mosaic. A hotlist allows a subject to mark locations that he or she feels will be useful 
for future reference. A URL in a hotlist defines both the Internet address and the type 
of server for a given WWW page. An example of a URL maintained in a hotlist would 
be "ftp://www.nps.navy.mil/". This is the File Transfer Protocol (FTP) server residing on 
"www.nps,navy.mil," the Internet Protocol(IP) address of the machine. In Table 5, the 
observer noted the total number of entries in each subject’s hotlist and used this to 
provide the average given for each TTM section. 

Table 5 also provides some statistical information, namely standard deviation and 
interval estimate, for this data. This information was calculated using the same method 
discussed in detail in Chapter II, under "Sample Size." 
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Yes 

No 

Average 

Hotlist 

sd 

i 

PM33 

5 

- 

26.0 

8.0 

7.84 

PM41 

4 

1 

8.4 

13.1 

12.84 

PM43 

4 

1 

2.8 

2.4 

2.35 

PM51 

1 

4 

0.0 

0.0 

0.0 


Table 5. Customized Environment 
sd=standard deviation 
i=interval estimate 

Subjects were also asked if they had access to a Unix based computer account at 
their home or previous command prior to arrival at the Naval Postgraduate School. All 
subjects stated that they did not have previous access. Again, these were the subject’s 
responses and were not independently verified by the observer. 

B. OBSERVATIONS 

The following data were gathered during the task observations conducted on ITM 
students at the Naval Postgraduate School. These groups of observations were designed 
to test the hypotheses discussed in the Methodology: 

Hypothesis 1. The abiUty to perform various tasks using the Internet and student 
Unix accounts improves with experience, experience being defined as time at the Naval 
Postgraduate School. 

Hypothesis 2. The prime source of the students’ ability to use the Internet and 
their Unix accounts is not through a formal source, such as class assignments, computer 
center lectures, or computer center personnel, but rather through an informal method of 
sharing knowledge among themselves. 
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The first four of the following tables (Tables 6 through 9); communications, data 
transfer, data location and cumulative results; are associated with hypothesis 1. These 
tables are based on direct behavioral observation by the observer, without input from the 
subject. The interval estimate for this group of observations was determined in Chapter 
II, under Sample Size." As discussed there, the sample data provided were within 53 
seconds of the actual ITM population with a confidence of 0.95. One column on Tables 
6 through 9 equals 60 seconds. Each ITM section’s results for each specific task are 
contained in Appendix B. Individual subject’s results are not provided. For each of the 
areas, the actual tasks used in the observation are contained in Appendix A. 


1. Communications 


Table 6 represents the cumulative totals for the four observed tasks associated with 
communication. All sections seemed to perform equally well on these tasks. As shown 
m the Literature review, these were the easiest tasks in the observation. Specifically, these 
communications tasks were: 

Al. Send an E-mail message. 

A2. Find a local (NFS) E-mail address. 

A3. Find an E-mail address within the same domain (.mil). 

A4. Find an E-mail address in an outside domain. 


A domain in this case is the top-level of an e-mail IP address, such as .edu or .mil. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

13 

- 

- 

- 

- 

7 

PM41 

9 

- 

1 

- 

- 

10 

PM43 

7 

1 

- 

- 


12 

PM51 

11 


_ 



9 


Table 6. Communication Tasks Performance 



















2. Data Transfer 


Table 7 represents the cumulative totals for the four observed tasks associated with 
data transfer. Performance stayed consistent at this point, except for an improvement in 
PM33 and a decrease in PM43. These tasks could also be considered less difficult than 
the data location tasks. Specifically, these data transfer tasks were: 

Bl. Perform basic file management (create, modify and delete files) within 
subject’s own account. 

B2. Download a file from a given FTP site. 

B3. Access a given Gopher site. 

B4. Access a given WWW site. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

11 

5 

1 

- 

- 

3 

PM41 

6 

5 

3 

- 

- 

6 

PM43 

4 

4 

1 

1 

■ - 

10 

PM51 

5 

5 

- 


- 

10 


Table 7. Data Transfer Tasks Performance 


3. Data Location 

Table 8 represents the cumulative totals for the four observed tasks associated with 
data location. These were the most difficult of the observed tasks. PM33 continued to 
show strong performance, while the other sections showed a decrease in performance. 
This was the task set that gave PM33 its overall edge, as shown in Table 9. Specifically, 
these data location tasks were: 
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Cl. Find general information on a given topic. 
C2. Find a specific piece of information. 

C3. Find a specific Gopher site. 

C4. Find a specific WWW site. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

8 

4 

5 

- 

1 

2 

PM41 

3 

1 

2 

- 

2 

12 

PM43 

2 

3 

2 

- 

- 

13 

PM51 

2 

4 

2 

1 

- 

11 


Table 8. Data Location Tasks Performance 


4. Cumulative Results for Hypothesis 1 

Hypothesis 1. The ability to perform various tasks using the Internet and student 
Unix accounts improves with experience, experience being defined as time at the Naval 
Postgraduate School. 

Table 9 shows the cumulative results of Tables 6 through 8. Two things appear 
significant with this table. First, PM43 performed poorer than all the other sections, 
including PM51. Second, PM33 out performed the other sections. These assumptions 
will be examined in detail in Chapter IV. 
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Table 9. Cumulative Task Performance 


5. Knowledge Source 


Table 10, knowledge source, is associated with the second hypothesis: 

Hypothesis 2. The prime source of the students’ ability to use the Internet and 
their Unix accounts is not through a formal source, such as class assignments, computer 
center lectures, or computer center personnel, but rather through an informal method of 
sharing knowledge among themselves. 

Table 10 represents the responses given by the subjects when asked to rate the top 
three sources for the knowledge they displayed in performing the previous sets of tasks. 
The responses were recorded as supplied by the subjects. They were not independently 
verified by the observer. 

The weighted total in Table 10 is given to provide a more intuitive means of 
comparison between the different sources of knowledge. The responses were weighted 
as follows: Responses rated 1st were given a weight of 3; responses rated 2nd were given 
a weight of 2; and responses rated 3rd were given a weight of 1. 

The 2nd and 3rd response rating columns do not equal the possible total of twenty 
because several subjects responded with null answers. 
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Table 10. Student’s Source of Knowledge 
(CC = Computer Center) 


Statistical analysis and interpretation of all observation data are provided in 
Chapter IV. Any raw data not presented here are given in Appendix B. 
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IV. ANALYSIS OF RESULTS 


This chapter will analyze the raw data reported in Chapter III, and determine if 
it supports the thesis hypotheses. It will also look at possible causes of any anomalies 
in the data. 

A. DEMOGRAPHICS 

The demographics alone do not provide support for any of the research questions 
of this thesis. They do, however, draw attention to some questions that arose during the 
examination of the data. 

The most puzzling question that arose concerns the performance and demographics 
of PM43. There seems to be a correlation between Table 4 and Table 11. Table 11 
shows PM43 performing worse than PM51 on the cumulative task results, which seems 
illogical due to PM43’s supposedly greater experience. Table 4 shows PM43 with much 
less account usage than all the sections, including PM5L There appeared to be some 
systematic variance in the data. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

32 

9 

6 

- 

1 

12 

PM41 

18 

6 

6 


2 

28 

PM43 

13 

8 

3 

1 


35 

PM51 

18 

9 

2 

1 

- 

30 

Total 

81 

32 

17 

2 

3 

105 


Table 11. Cumulative Task Performance 
ITM Section versus Time 
(N/A = Not Able to Perform Task) 
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There were two possible causes discovered for this problem with PM43. One is 
a sampling error on the part of the observer and the other is an uncontrolled outside 
influence upon PM51, which gives the mistaken impression of a problem in PM43’s 
performance. The sample taken was not random. Instead it relied on volunteers. In 
the case of PM43, one person facilitated the recruitment of volunteers for the 
observations. If this person used acquaintances with the same social and educational 
backgrounds, and if this background was conducive to poor performance on the task 
observations, then this sample bias may be the source of the problem. 

The other possible source of the inconsistency is a systematic variance in PMSl’s 
data. In other words, PM43’s performance was not sub-par, but instead PMSl’s 
performance was superior. Upon investigation, it became known that PM51 did receive 
outside assistance in learning about the Internet. Several students from PM31, a previous 
graduated ITM section, remained on campus while awaiting orders. One of these 
students, through personal connections, assisted PM51 in getting started with their Unix 
accounts. PM51 also showed an increased response towards formal methods, as shown 
in Appendix B. These factors combined may have caused PMSl’s ability to increase. 
The problem, therefore, was not a problem with PM43, but rather an over achievement 
by PM51. This is believed to be the actual cause of the observed variance in the data. 

The demographics showed no other problems in the data. Account customization 
was consistent with the hypothesis that the more experienced sections were more 
sophisticated in there Internet usage. 

B. HYPOTHESIS 1 


Hypothesis 1. The ability to perform various tasks using the Internet and student 
Unix accounts improves with experience, experience being defined as time at the Naval 
Postgraduate School. 


30 


An Analysis of Variance (ANOVA) was used to test hypothesis 1. An ANOVA 
is used to test if the means of several sample sets are statistically equal. It compares the 
effect of different independent variables on the dependent variable.(Emory & Cooper, 
1991) In this case, the time taken by a task was used as the dependent variable, while 
ITM section was the independent variable. 

One problem involved with using the ANOVA was how to represent the "Not 
Able" or "N/A" responses for the subjects who took longer than five minutes. "N/A" 
must somehow be presented as interval data. There seemed to be two possible ways to 
do this. 

One option would be to give "N/A" an arbitrary numerical value based on some 
quantity such as mean time for receiving help from a computer center consultant. The 
problem with this method was determining an equitable numerical value to assign to 
"N/A." It was felt that the "N/A" response could be any length of time from slightly 
more than five minutes to possibly indefinite. This option was deemed unreliable and 
unscientific. 

The second option was based on the bipolar distribution of the data. As shown 
in Table 12, most measurable responses occurred early in the task observation. This 
shows that for most subjects, they either could perform the task or they could not. In 
other words, they were not able to work through the task if they did not immediately 
know the procedure. This dichotomy presented the opportunity to assign integer values 
to whether the subject was able to perform the task or not. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

Total 

81 

32 

17 

2 

3 

105 


Table 12. Cumulative Response Time 
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Therefore, for the ANOVA test of hypothesis 1 presented in Table 13, subjects 
who were able to perform the tasks were assigned a value of one. Subjects who were 
unable to perform the task were assigned a value of zero. 

The null hypothesis(Ho) used to check hypothesis 1 is that the task performance 
of each section is statistically equal to the task performance of the other sections. Table 
13 shows the results of the ANOVA for each task category plus for the cumulative. 

Null Hypothesis: 

HqI P33=P41=P43=P5i 

where: P = performance on given task set 

subscript = TTM section 


Tasks 

Sum of 

DF 

Mean 

F 

Sig of 


Squares 


Squares 


F 

Communications 

.650 

3 

.217 

.853 

.469 

Data Transfer 

1.737 

3 

.579 

2.628 

.056 

Data Location 

3.850 

3 

1.283 

6.058 

.001 

Cumulative 

4.946 

3 

1.649 

7.190 

.000 


Table 13. ANOVA of Task Results 


Using p<.05 as the cutoff point, we failed to reject the null hypothesis in the case 
of communications and data transfer. This means that all four TTM sections performed 
statistically equal on these tasks. In the case of data location, however, we did reject the 
null hypothesis, showing that the different ITM sections were not equal. 

The tools used with information location are more complex than those used for 
communications and data transfer. This is consistent with failing to reject the null 
hypothesis for communications and data transfer, but rejecting it for data location. 
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We also rejected the null hypothesis for the cumulative data. While this supports 
our hypothesis, the cumulative calculations also had the advantage of a data set three 
times as large as the non-cumulative tasks. The increased size pf the data set made even 
minor variations statistically significant. 

C. HYPOTHESIS 2 

Hypothesis 2. The prime source of the students’ ability to use the Internet and 
their Unix accounts is not through a formal source, such as class assignments, computer 
center lectures, or computer center personnel, but rather through an informal method of 
sharing knowledge among themselves. 

Statistical analysis was performed on the data using a Friedman Two-Way 
ANOVA. A Friedman Two-Way ANOVA examines the variance of ranked data within 
each block. A block in this case was one subject. It is based on two assumptions. One, 
that the results of one subject do not influence any other subject. Two, that each 
subject’s data is arranged in a rank order. Both of these criteria were met by the 
observation data for hypothesis 2. (Conover, 1971) 

Again, there was a problem with null data. The subjects were asked to rank only 
their top three choices out of a possible six choices. It was decided to give all null data 
the rank of four. Therefore, each block of data would have the top three sources, as 
ranked by the subjects, and all remaining options would be given the rank of four. The 
Friedmcin Two-Way ANOVA is tolerant of ties in the ranked data. (Conover, 1971) 

The Friedman Two-Way ANOVA provides two pieces of information. One, a 
mean rank for each variable, and two, a significance factor for testing a null hypothesis 
of equality. 

The null hypothesis used for hypothesis 2 was that all sources of ability were 
ranked equal. Table 14 shows the results of the Friedman Two-Way ANOVA testing the 
following null hypothesis. 
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Null Hypothesis(Ho): 


Hq- Si“S 2—S3—S4—S5—Sg 

where: subscript = specific source as shown in Table 15. 



DF 

Sig 

32.6857 

5 

.000 


Table 14. Friedman Two-Way ANOVA Results 


Using a p<.05 cutoff, we were able to reject the null hypothesis. Therefore, the 
sources that subjects reported for where they learned the Internet were not statistically 
equal. 

The rest of the information provided by the Friedman Two-Way ANOVA is given 
in Table 15. This gives us the mean rank of the data over all subjects. 


S„ 

Source 

Mean 

Rank 

1 

CC 

Personnel 

4.47 

2 

CC Class 

4.47 

3 

CC 

Handout 

4.32 

4 

On Own 

2.15 

5 

Classmates 

2.40 

6 

Assignment 

3.17 

- 

Other 

- 


Table 15. Mean Rank of Knowledge Sources 
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Overall, this shows that "On Own" and "Shown by Classmates" are the common 
source of learning Internet techniques. These can be considered the informal techniques, 
thus supporting the hypothesis that students are learning in informal ways. The only 
formal method worth mentioning is "class assignments" which performed far better than 
the Computer Center related sources. The results of each individual section are shown in 
Appendix C. 

The one exception is PMSl’s individual results, as shown in Appendix C, which 
show a stronger use of formal methods. This is the only section which did not rank "on 
own" and "shown by classmates" as the top two sources. PM51, instead, ranked "class 
assignments" as their top choice. This is assumed to be the result of more classroom 
instructors beginning to including the Internet in their class materials. 

The consequences of these observations will be discussed in Chapter V , 
"Conclusions and Recommendations." 


35 




36 



V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

1. Study Benefits 

This thesis has portrayed the current state of Internet usage by ITM students at the 
Naval Postgraduate School. ITM students are expected to function as the future DoD IT 
managers. In this role they will need to be literate in the use of the Internet. It has 
shown the need for earlier direction in teaching the Internet to ITM students. 

As a secondary benefit, the data recorded in this thesis may also be useful in 
future research as both a starting point and a baseline for comparison against new data. 

2. Research Questions 

Is there a relationship between experience and the ability to use the Internet 
effectively? The answer is a definite yes. This in itself is not remarkable, and in fact 
seems quite obvious. The interesting revelation is the characteristics of the relationship. 

Only PM33’s performance could be considered remotely adequate, and even PM33 
was only able to perform 48 out 60 tasks, or 80%. The other sections performance 
should be considered substandard. This is not necessarily a major problem. Students, as 
shown by PM33, are becoming Internet literate before they leave NPS. They are not, 
however, receiving the full benefit of Internet usage during their entire course of study 
at NPS. 

What instructional source has taught ITM students to use the Internet? The overall 
source of instruction is very informal. There is certainly a correlation between this fact 
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and the low scores show by all sections except PM33. Informal learning methods are 
likely to be more inefficient than formal ones. 

There is however, as shown in Table 16, a migration towards more formal 
methods taking place. This should be considered a positive sign as shown by PMSl’s 
advanced performance. 


S„ 

Source 

PM33 

PM41 

PM43 

PM51 

1 

CC Personnel 

4.60 

4.00 

4.70 

4.60 

2 

CC Class 

4.60 

4.80 

3.90 

4.60 

3 

CC Handout 

4.20 

4.20 

4.70 

4.20 

4 

On Own 

1.20 

2.00 

2.50 

3.20 

5 

Classmates 

2.20 

2.40 

2.00 

2.60 

6 

Assignment 

4.20 

3.60 

3.20 

1.80 


Table 16. Source Rank by Section 


B. RECOMMENDATIONS 

The following recommendations are designed to improve the ability of ITM 
students to use the Internet, based on the results of the research questions. 

Based on the research conducted in this thesis, formal classroom instruction is 
needed to teach ITM students the Internet. This conclusion is based on the current 
informal methods used and the low observation performance they produced. If a fo rmal 
class is impracticable, then an online tutorial may be appropriate. The advantages and 
disadvantages of both options should be examined in detail as possible future research. 

Currently, the majority of the students learn through informal methods. Non- 
mandatoiy computer center classes are available to all these students. They are not being 
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used. If this status quo continues, without the adoption of a formal Internet class, then 
an online tutorial would be the appropriate instructional tool. Stimulating usage will most 
likely continue to be a drawback with any non-mandatory method. 

However, it has also been shown that there is a gradual shift taking place towards 
formal instruction as shown by PMSl’s results. If it is desired for this trend to continue, 
then a formal Internet class would be the appropriate solution. A formal class would 
follow a standard teaching methodology, which would provide a much greater chance of 
long term comprehension than informal methods. 

1. Online Tutorial 

Constructing an online tutorial should almost certainly be considered. Even if a 
formal class structure is adopted, a ready reference source would be invaluable until the 
students became more sophisticated in customizing there own environment and tools. 

The most useful form for a tutorial would be a WWW based page on a server 
controlled by the ITM curriculum, or at least by the Systems Management curriculum. 

An online tutorial also has the benefit of portability. With the increased use of 
the Internet within DoD, this could have several advantages. A well designed tutorial 
could be exported to other curriculums and DoD activities with WWW capabilities. Also, 
if there were any interest in standardizing what resource discovery tools and services were 
used within DoD, a canned tutorial could be a way to pubUcize them. 

2. Class Instruction 

The preferred means of teaching the Internet, however, would be the formal 
classroom instruction. A basic Internet skills curriculum would not occupy enough 
material to fill an entire quarter course. It would be better taught as a portion of an 
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introductory course such as IS2000 or as an independent half quarter course during the 
refresher quarter. 

Formal classroom instruction also has the advantage of being mandatory. 
Hopefully, mandatory instruction will not be necessary once the student sees the 
advantages of using the Internet, but until that time, class assignments would point the 
student in the right direction. 

C. FURTHER RESEARCH 


Based on the above recommendations, follow-on research into designing a tutorial 
or class may be useful. Further testing to verify the results of these implementations 
would then also be useful. 

1. Design the online tutorial to be used. This would require a strong 
familiarization with hypertext mark-up language (html) and its associated use on WWW 
servers. It would also require research in tutorial design methods. Finally, the material 
for constructing the tutorial would have to be gathered and verified. 

2. Design the class syllabus for an Internet course. Again, this would require the 
gathering and verification of the class material. This material would then have to be 
placed into a logical sequence, and the associated teaching media developed. 

3. Test the effectiveness of whatever new techniques are implemented. This would 
consist of reconducting the observations of this thesis. These new results would be 
compared to this thesis’s baseline results. This would confirm the effectiveness of 
whatever new instruction methods were chosen. Even if no new methods were 
implemented, tracking the growing use of the Internet in other ITM classes would at least 
confirm the results shown by PM51. 
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APPENDIX A. OBSERVATION TASKS 


Al. Send E-mail to the following address: afmusgro@nps.navy.mil 
Subject: Test Body: This is a test. 

A2. William J. Haga is a faculty member at NFS, find his e-mail address. 

A3. Bill Doe is in the Army, location unknown, find his e-mail address. 

A4. Karen Jones works at the University of Alaska at Fairbanks, find her e-mail address. 


Bl. Create a file "test", move it to any other directory, then delete it. 

B2. Download the file "newnames.txt: from the following FTP site: 

site: byrd.mu.wvnet.edu login: anonymous 

directory: pub/history/militaryAJSN 

B3. Access the following gopher site: 
gopher.ucr.ac.cr 

B4. Access the following WWW page: 
http://wsk.eit.com/ 

Cl. Find information on the subject "alienation". 

C2. Find a copy of Sir Walter Scott’s book Ivanhoe . 

C3. Find the following gopher site: 

PATHY Gopher for Medical Information 

C4. Find the following WWW page: 

EUNet Norway Information Services 

For the above tasks, where did you gain your knowledge (rank the top three): 

a. Shown by Computer Center Personnel 

b. Attended Computer Center Class 

c. Read Computer Center Handout/Bulletin 

d. On your own/exploring 

e. Shown by classmates 

f. From class assignments in a required course 

g. Other (please specify) 
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APPENDIX B. OBSERVATION RAW DATA 


Demographics 

Hotlist Size by Subject and Section 








AVG 

PM33 

39 

20 

27 

19 

25 

26.0 

PM41 

0 

12 

0 

0 

30 

8.4 

PM43 

9 

2 

0 

0 

3 

2.8 

PM51 

0 

0 

0 

0 

0 

0.0 


Task Observations 

For ease of visual examination all "0" responses have been replaced with a 


Communications 

Al. Send an E-mail message. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

5 

- 

- 

- 

- 

- 


5 

- 

- 

- 

- 

- 

PM43 

3 

1 

- 

- 

- 

- 


5 

- 

- 

- 

- 

- 
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Communications (continued) 

A2. Find a local E-mail (NFS) address. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

5 

- 

- 

- 

- 

- 


2 

- 

- 

- 

- 

3 

PM43 

3 

- 

- 

- 

- 

2 

PM51 

5 

- 

- 

- 


- 


A3. Find an E-mail address within the same domain (.mil). 


IHHI 

<1 

1-2 

2-3 

3-4 

4-5 

N/A 


3 

- 


- 

- 

2 

PM41 

2 

- 

- 

- 

- 

3 


1 

- 

- 

- 

- 

4 

PM51 

1 

- 

- 

- 

- 

4 


A4. Find an E-mail address in an outside domain. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

- 

- 

- 

- 

- 

5 

PM41 

- 

- 

1 

- 


4 

PM43 

- 

- 


- 

- 

5 

PM51 

- 

- 

- 

- 

- 

5 
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Data Retrieval 


Bl. Perform basic file management. 



B2. Download a file from a given FTP site. 

<1 1-2 2-3 3-4 4-5 N/A 


PM33 - 4 1 


PM41 - 2 ... 3 


PM43 1 1 1 - - 2 


PM51 - 4 ... 1 


B3. Access a given Gopher site. 



45 


























































Data Retrieval (continued) 

B4. Access a given WWW site. 



<1 

1-2 

2-3 

PM33 

5 

- 

- 

PM41 

3 

1 

1 

PM43 

3 

- 

- 

PM51 

2 

- 

- 


Data Location 

Cl. Find general information on a given topic. 



<1 

1-2 

2-3 

PM33 

3 

- 

2 

PM41 

1 

- 

1 

PM43 

1 

- 

2 

PM51 

1 

2 

1 


C2. Find a specific piece of information. 



<1 

1-2 

2-3 

PM33 

3 

- 

1 

PM41 

1 

- 

- 

PM43 

1 

1 

- 

PM51 

1 

2 

- 
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C3. Find a specific Gopher site. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

- 

2 

1 

- 

1 

1 


- 

- 

- 

- 

2 

3 

PM43 

- 

1 

- 

- 

- 

4 

PM51 

- 

- 

- 

- 

- 

5 


C4. Find a specific WWW site. 



<1 

1-2 

2-3 

3-4 

4-5 

N/A 

PM33 

2 

2 

1 

- 

- 

- 

PM41 

1 

2 

1 

- 

- 

2 

PM43 

- 

1 

- 

- 

- 

4 

PM51 

- 

1 

1 

- 

- 

4 
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Source of Knowledge by Section 


CC = Computer Center 
Weighted Total: 1st = 3 

2nd = 2 
3rd= 1 


PM33 


Source 

1st 

2nd 

3rd 

Total 

Weighted 

Total 

CC 

Personnel 

- 

- 

- 

0 

0 

CC Class 

- 

- 

- 

0 

0 

CC 

Handout 

- 

- 

1 

1 

1 

On Own 

4 

1 


5 

14 

Classmates 

1 

3 

- 

4 

9 

Assignment 

- 


1 

1 

1 

Other 

- 

- 


0 

0 
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Source of Knowledge by Section (continued) 


PM41 




Source 


CC 

Personnel 


CC Class 


CC 

Handout 


On Own 


Classmates 


Assignment 


Other 


Weighted 

Total 
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APPENDIX C. FRIEDMAN TWO-WAY ANOVA RESULTS 

Mean Ranks by Section 


S„ 

Source 

PM33 

PM41 

PM43 

PM51 

1 

CC Personnel 

4.60 

4.00 

4.70 

4.60 

2 

CC Class 

4.60 

4.80 

3.90 

4.60 

3 

CC Handout 

4.20 

4.20 

4.70 

4.20 

4 

On Own 

1.20 

2.00 

2.50 

3.20 

5 

Classmates 

2.20 

2.40 

2.00 

2.60 

6 

Assignment 

4.20 

3.60 

3.20 

1.80 


t 

DF 

Sig 

14.8286 

5 

.0111 


Source 


CC Personnel 


CC Class 


CC Handout 


On Own 


Classmates 


Assignment 


Other 


Mean 

Rank 










































































DF 


S„ 

Source 

1 

CC Personnel 

2 

CC Class 

3 

CC Handout 

4 

On Own 

5 

Classmates 

6 

Assignment 

- 

Other 


Mean 

Rank 



DF 

Sig 

9.1143 

5 

.1046 








































PM43 (Continued) 


S„ 

Source 

Mean 

Rank 

1 

CC Personnel 

4.70 

2 

CC Class 

3.90 

3 

CC Handout 

4.70 

4 

On Own 

2.50 

5 

Classmates 

2.00 

6 

Assignment 

3.20 

- 

Other 

- 


PM51 



DF 

Sig 

9.5714 

5 

.0883 


S 


Source 



CC Personnel 
CC Class 
CC Handout 
On Own 
Classmates 
Assignment 


Mean 

Rank 

4.60 

4.60 

4.20 

3.20 

2.60 


6 


1.80 
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